Papers by Mohammed Saidul Islam
BanglaRQA: A Benchmark Dataset for Under-resourced Bangla Language Reading Comprehension-based Question Answering with Diverse Question-Answer Types (2022.findings-emnlp)
Copied to clipboard
Syed Mohammed Sartaj Ekram, Adham Arik Rahman, Md. Sajid Altaf, Mohammed Saidul Islam, Mehrab Mustafy Rahman, Md Mezbaur Rahman, Md Azam Hossain, Abu Raihan Mostofa Kamal
| Challenge: | a lack of diverse and comprehensive question-answering datasets exists in under-resourced languages like Bangla. |
| Approach: | They propose a reading comprehension-based Bangla question-answering dataset . the dataset includes answerable and unanswerable questions covering four categories of questions . |
| Outcome: | The proposed dataset shows that it performs well as a training resource in high-resource languages. |
Are Large Vision Language Models up to the Challenge of Chart Comprehension and Reasoning (2024.findings-emnlp)
Copied to clipboard
Mohammed Saidul Islam, Raian Rahman, Ahmed Masry, Md Tahmid Rahman Laskar, Mir Tafseer Nayeem, Enamul Hoque
| Challenge: | Recent studies have demonstrated that large vision language models (LVLMs) are not multi-modal and lack multi-tasking capabilities. |
| Approach: | They evaluate the performance of large vision language models (LVLMs) for chart understanding and reasoning tasks and compare them to open-source models. |
| Outcome: | The proposed models demonstrate impressive abilities in generating fluent texts covering high-level data insights, but they also encounter common problems like hallucinations, factual errors, and data bias. |
Judging the Judges: Can Large Vision-Language Models Fairly Evaluate Chart Comprehension and Reasoning? (2025.acl-industry)
Copied to clipboard
Md Tahmid Rahman Laskar, Mohammed Saidul Islam, Ridwan Mahbub, Ahmed Masry, Mizanur Rahman, Amran Bhuiyan, Mir Tafseer Nayeem, Shafiq Joty, Enamul Hoque, Jimmy Huang
| Challenge: | Large Vision-Language Models (LVLMs) are expensive and time-consuming to evaluate . however, they are limited in their use in industrial settings due to their limited availability and limited resources. |
| Approach: | They evaluate 13 open-source LVLMs as judges for diverse chart comprehension and reasoning tasks. |
| Outcome: | The proposed models can be used to assess chart comprehension and reasoning tasks, but they are expensive and time-consuming. |
DataNarrative: Automated Data-Driven Storytelling with Visualizations and Texts (2024.emnlp-main)
Copied to clipboard
| Challenge: | Data-driven storytelling uses visual aids and visualizations to convey insights. |
| Approach: | They propose a task for data story generation using large language models and a benchmark containing 1,449 stories from diverse sources. |
| Outcome: | The proposed framework outperforms non-agentic counterparts in both model-based and human evaluations, but also reveals unique challenges in data story generation. |
From Charts to Fair Narratives: Uncovering and Mitigating Geo-Economic Biases in Chart-to-Text (2025.emnlp-main)
Copied to clipboard
Ridwan Mahbub, Mohammed Saidul Islam, Mir Tafseer Nayeem, Md Tahmid Rahman Laskar, Mizanur Rahman, Shafiq Joty, Enamul Hoque
| Challenge: | Existing VLMs produce more positive descriptions for high-income countries compared to middle- or low-income nations, even when country attribution is the only variable changed. |
| Approach: | They propose to automate the process by generating textual summaries of charts using vision-language models to understand how a country’s economic status influences the sentiment of generated summary. |
| Outcome: | The proposed model amplifys geo-economic biases in 6,000 chart-country pairs from six widely used vision-language models to understand how a country’s economic status influences the sentiment of generated summaries. |
Deploying Tiny LVLM Judges for Real-World Evaluation of Chart Models: Lessons Learned and Best Practices (2025.emnlp-industry)
Copied to clipboard
Md Tahmid Rahman Laskar, Mohammed Saidul Islam, Ridwan Mahbub, Mizanur Rahman, Amran Bhuiyan, Israt Jahan, Mir Tafseer Nayeem, Shafiq Joty, Enamul Hoque, Jimmy Huang
| Challenge: | Large Vision-Language Models (LVLMs) with only 7B parameters perform poorly as judges in resource-constrained settings. |
| Approach: | They propose two approaches to ensure costefficient evaluation by combining multiple criteria into a single query and domainadaptive transfer learning to create a 2Bparameter VLM on a chart dataset. |
| Outcome: | The proposed model can effectively transfer knowledge from one dataset to another to make it a more specialized model. |
DashboardQA: Benchmarking Multimodal Agents for Question Answering on Interactive Dashboards (2026.findings-eacl)
Copied to clipboard
Aaryaman Kartha, Ahmed Masry, Mohammed Saidul Islam, Thinh Lang, Shadikur Rahman, Ridwan Mahbub, Mizanur Rahman, Mahir Ahmed, Md Rizwan Parvez, Enamul Hoque, Shafiq Joty
| Challenge: | Existing question-answering benchmarks for data visualizations focus on static charts instead of interactive dashboards. |
| Approach: | They propose a benchmark to assess how vision-language GUI agents comprehend and interact with real-world dashboards. |
| Outcome: | The first benchmark explicitly designed to assess how vision-language GUI agents comprehend and interact with real-world dashboards. |
Lost in Translation: Do LVLM Judges Generalize Across Languages? (2026.findings-acl)
Copied to clipboard
Md Tahmid Rahman Laskar, Mohammed Saidul Islam, Mir Tafseer Nayeem, Amran Bhuiyan, Mizanur Rahman, Shafiq Joty, Enamul Hoque, Jimmy Huang
| Challenge: | MM-JudgeBench is the first large-scale benchmark for multilingual and multimodal judge model evaluation. |
| Approach: | They propose a multilingual benchmark for multilingual and multimodal judge model evaluation that includes over 60K pairwise preference instances spanning 25 typologically diverse languages. |
| Outcome: | The proposed benchmark includes over 60K pairwise preference instances spanning 25 languages. |
Aligning Text, Code, and Vision: A Multi-Objective Reinforcement Learning Framework for Text-to-Visualization (2026.eacl-long)
Copied to clipboard
| Challenge: | Text2Vis systems generate functional code but resulting charts lack semantic alignment and clarity. |
| Approach: | They propose a framework that integrates post-execution feedback with textual accuracy, code validity, and visualization quality. |
| Outcome: | The proposed framework outperforms strong zero-shot and supervised baselines and shows robust generalization to out-of-domain datasets. |
ChartQAPro: A More Diverse and Challenging Benchmark for Chart Question Answering (2025.findings-acl)
Copied to clipboard
Ahmed Masry, Mohammed Saidul Islam, Mahir Ahmed, Aayush Bajaj, Firoz Kabir, Aaryaman Kartha, Md Tahmid Rahman Laskar, Mizanur Rahman, Shadikur Rahman, Mehrad Shahmohammadi, Megh Thakkar, Md Rizwan Parvez, Enamul Hoque, Shafiq Joty
| Challenge: | Chart Question Answering systems are limited in their ability to interpret data visually and reason with visual representations. |
| Approach: | They propose a chart-based chart question-answering system that includes 1,341 charts from 99 diverse sources and 1,948 questions in various types. |
| Outcome: | The new benchmark includes 1,341 charts from 99 diverse sources and 1,948 questions in various types. |
BenLLM-Eval: A Comprehensive Evaluation into the Potentials and Pitfalls of Large Language Models on Bengali NLP (2024.lrec-main)
Copied to clipboard
Mohsinul Kabir, Mohammed Saidul Islam, Md Tahmid Rahman Laskar, Mir Tafseer Nayeem, M Saiful Bari, Enamul Hoque
| Challenge: | Large Language Models (LLMs) have emerged as one of the most important breakthroughs in natural language processing. |
| Approach: | They propose to evaluate LLMs in Bengali to benchmark their performance . they select Bangla NLP tasks such as text summarization, question answering, paraphrasing . |
| Outcome: | The proposed model performs better in some tasks than current models, but in most tasks, it is poor . |